Skip to content

[AgentX][B300] Dsv4 update with dspark - #3606

Open
Oasis-Git wants to merge 3 commits into
mainfrom
config/dsv4-b300-dspark-coordination
Open

Oasis-Git wants to merge 3 commits into
mainfrom
config/dsv4-b300-dspark-coordination

Conversation

@Oasis-Git

Copy link
Copy Markdown
Collaborator

更新 B300 DSV4 DSpark 配置,启用 DP 推测解码与预填充协调,保留 TP4/TP8 低延迟测试点。

更新 B300 DSV4 DSpark 配置,启用 DP 推测解码与预填充协调,保留 TP4/TP8 低延迟测试点。
@github-actions

Copy link
Copy Markdown
Contributor

Thanks for the contribution!

  • Review: If this PR changes files owned by someone other than a repository admin or @SemiAnalysisAI/core, ask one eligible CODEOWNER to complete the latest PR_REVIEW_CHECKLIST.md before contacting a core maintainer on Slack. Follow the template exactly, including As a PR reviewer and CODEOWNER, I have reviewed this and have, so sign-off verification triggers.
  • PR verification: Sweeps only run on labeled PRs. Add full-sweep-fail-fast (strongly recommended); use full-sweep-enabled only when matrix jobs should continue after a failure.
  • After merging: PR authors must ensure all GitHub Actions jobs pass. Transient failures often pass on rerun; see how to rerun failed jobs.
中文

感谢你的贡献!

  • **审阅:**如果 PR 修改的文件归属于仓库管理员及 @SemiAnalysisAI/core 之外的 CODEOWNER,请先联系一位有资格的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md,再通过 Slack 联系核心维护者。必须严格遵循模板,并保留 As a PR reviewer and CODEOWNER, I have reviewed this and have,才能触发签核验证。
  • **PR 验证:**扫描仅在带有标签的 PR 上运行。强烈建议添加 full-sweep-fail-fast;仅当需要矩阵任务在失败后继续运行时才使用 full-sweep-enabled。
  • **合并后:**PR 作者必须确保所有 GitHub Actions 任务通过。临时性失败通常可以通过重新运行恢复;参见重新运行失败任务的说明。

@Oasis-Git Oasis-Git added the full-sweep-fail-fast Full sweep with canary gate; first failure cancels the rest of that matrix (recommended) label Sep 30, 2026
补充 DSpark 提交的变更日志 PR 链接,修复矩阵规划校验失败。

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Beyond the inline finding, I checked the disable-circuit-breaker addition to the shared DEP8 router config and the max-running-requests cut to 768 for c512/c576 — both read as deliberate tuning tradeoffs rather than a missing safety guard or admission regression. I also confirmed the container image digest matches exactly between agentic.yaml and nvidia-master.yaml, and that none of the new env vars hardcode a synthetic acceptance length.

Extended reasoning...

The diff bumps the SGLang container image and reworks scheduling/memory/env knobs for the DSV4 B300 DSpark benchmark across the recipe, master config, and changelog; it touches no auth/crypto/permission surface. A confirmed inline finding already flags a missing required pr-link in the new changelog entry, which should block merge until fixed. I additionally verified image-digest parity between the recipe and master config (an AGENTS.md invariant) and ruled out two candidate concerns (circuit-breaker removal on the shared DEP8 router, reduced max-running-requests for c512/c576) as intentional tuning rather than bugs, and found no reintroduction of hardcoded synthetic acceptance lengths.

Comment thread inferencex-e2e/perf-changelog.yaml Outdated
Comment on lines +9131 to +9137
- config-keys:
- dsv4-fp4-b300-sglang-agentic-hicache-mtp
scenario-type:
- agentic-coding
description:
- "Update B300 DSV4 DSpark performance with DP spec/prefill coordination, mixed 64K chunks and interval 32; retain TP4/TP8 low-latency coverage."
- "更新 B300 DSV4 DSpark 性能,启用 DP 推测解码与预填充协调、64K 混合分块和间隔 32;保留 TP4/TP8 低延迟测试点。"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 The new changelog entry for dsv4-fp4-b300-sglang-agentic-hicache-mtp is missing the required pr-link: field, which every other entry in the file has. AGENTS.md requires this field, and infx/workflows/validate_perf_changelog.py's compare_entries() calls validate_added_pr_link(str(entry.get("pr-link") or ""), pr_number) on every newly added entry, raising ChangelogValidationError for an empty/missing link. Fix: append pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/<N> (or the XXX placeholder for PR runs) to this entry, matching every other block in the file.

Why this was flagged

The entry appended at inferencex-e2e/perf-changelog.yaml:9131-9137 (config-keys: dsv4-fp4-b300-sglang-agentic-hicache-mtp) has description lines but no pr-link: key. validate_perf_changelog.py's compare_entries() (called from the CI changelog check) iterates additions and calls validate_added_pr_link(str(entry.get("pr-link") or ""), pr_number); with the key absent this evaluates to an empty string, which fails CANONICAL_PR_LINK.fullmatch(link) (main-branch run) or the pr_number equality check (PR run), raising ChangelogValidationError. A repo-wide count confirms every other of the file's 1057 config-keys blocks has a matching pr-link line (1056 pr-link lines total), so this is the sole entry missing it, and the merge/CI gate for this PR will fail until it is added.

Verification: normal. The appended entry at inferencex-e2e/perf-changelog.yaml:9131-9137 (config-keys: dsv4-fp4-b300-sglang-agentic-hicache-mtp) contains only config-keys/scenario-type/description and no pr-link: key, whereas every other entry has one (lines 9110, 9122, 9129). validate_perf_changelog.py wires this into CI (.github/workflows/run-sweep.yml). compare_entries() collects additions = head_entries[len(base_entries):] and for each calls validate_added_pr_link(str(entry.get("pr-link") or ""), pr_number) (line 193).

@github-actions

github-actions Bot commented Sep 30, 2026 •

Copy link
Copy Markdown
Contributor

更新 B300 DSpark 镜像至 2026-10-08 CUDA 13 开发版 nightly,合并最新 main 并解决变更日志冲突。
@nvpohanh

nvpohanh commented Oct 9, 2026

Copy link
Copy Markdown
Collaborator

[by Claude Code] The SGLang cookbook integration of this PR's AgentX config (dsv4-fp4-b300-sglang-agentic-hicache-mtp) will happen in sgl-project/sglang#43083, in the Agentic Long-Context Deployment section of the DeepSeek-V4 cookbook page. This PR has no authorized /use <run_id> comment yet, so the cookbook shows the PR-head recipe as "in progress" until a measured run is pinned.

@nvpohanh

nvpohanh commented Oct 9, 2026

Copy link
Copy Markdown
Collaborator

[by Claude Code] sgl-project/sglang#43083 has been merged. This PR's AgentX config (dsv4-fp4-b300-sglang-agentic-hicache-mtp) is now in the Agentic Long-Context Deployment section of the SGLang DeepSeek-V4 cookbook: https://docs.sglang.io/cookbook/autoregressive/DeepSeek/DeepSeek-V4#agentic-long-context-deployment

This PR has no authorized /use <run_id> comment yet, so the cookbook shows the recipe at this PR's head (5b394d395) as "in progress". It switches to the measured run at the next cookbook sync after a run is pinned or this PR merges.

@nvpohanh

nvpohanh commented Oct 9, 2026

Copy link
Copy Markdown
Collaborator

[by Claude Code] @Oasis-Git The full sweep for this PR is blocked by the new TP4 DSpark lane: the server hangs at startup every time (3 of 3 attempts). Could you debug it?

Symptom. The fail-fast canary is override_tp4_c1 (TP4, DSpark block 6, flashinfer_mxfp4, DeepSeek-V4-Pro-0813). All checkpoint shards finish loading, then the server prints nothing until the 7200 s health timeout kills it. There is no traceback, OOM or NCCL error. The canary fails, so fail-fast skips every other point.

Run Image Node Result
36687449966 dev-cu13-nightly-0929 b300 gpu-00 Hung after 66/66 shards
37841237652 attempt 1 nightly-dev-cu13-20261008-7760d526 b300 gpu-01 Hung after 66/66 shards
37841237652 attempt 2 nightly-dev-cu13-20261008-7760d526 b300 gpu-00 Hung after 66/66 shards (rerun)

Where it stops. In the merged EAGLE TP4 recipe (b300-fp4-eagle/agentic.yaml, base DeepSeek-V4-Pro, almost the same args), DeepSeek V4 MHC prenorm prewarm: ... n_splits buckets is logged about 1 s after the shards finish loading. In these DSpark/0813 TP4 runs, that line never appears. In SGLang 7760d526, this places the hang in DeepseekV4ForCausalLM.load_weights, after the shard iterator ends and before _prewarm_mhc_kernels logs. That covers draining the async weight-loader futures and post_load_weights. TP4 is new in this PR (main only has TP8 for this config), and no merged SGLang config runs DeepSeek-V4-Pro-0813 + DSpark at TP4.

Suggestions.

  • Reproduce TP4 c1 directly and take a py-spy dump of all four TP ranks once loading stalls.
  • If TP4 can't be fixed quickly, drop the TP4 lane (the search-space entry plus the override_tp4_* blocks) and update the changelog text, so the TP8 and DEP8 points can be measured.

The server logs are in each run's server_logs_dsv4_tp4_conc1_* artifact.

@nvpohanh

Copy link
Copy Markdown
Collaborator

[by Claude Code] The SGLang cookbook update of this PR's AgentX config (dsv4-fp4-b300-sglang-agentic-hicache-mtp) will happen in sgl-project/sglang#43983, in the Agentic Long-Context Deployment section of the DeepSeek-V4 cookbook page. The config has been live in that section since sgl-project/sglang#43083; sgl-project/sglang#43983 refreshes it with the pin below.

This PR has no authorized /use <run_id> comment yet, so the cookbook shows the recipe at this PR's head (5b394d395) as "in progress". It switches to the measured run at the next cookbook sync after a run is pinned or this PR merges.

@nvpohanh

Copy link
Copy Markdown
Collaborator

[by Claude Code] sgl-project/sglang#43983 has been merged. This PR's AgentX config (dsv4-fp4-b300-sglang-agentic-hicache-mtp) is now in the Agentic Long-Context Deployment section of the SGLang DeepSeek-V4 cookbook: https://docs.sglang.io/cookbook/autoregressive/DeepSeek/DeepSeek-V4#agentic-long-context-deployment

This PR has no authorized /use <run_id> comment yet, so the cookbook shows the recipe at this PR's head (5b394d395) as "in progress". It switches to the measured run at the next cookbook sync after a run is pinned or this PR merges.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

full-sweep-fail-fast Full sweep with canary gate; first failure cancels the rest of that matrix (recommended)

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

2 participants